Test_V2

From: https://osf.io/p5xsd/files/osfstorage

knitr::opts_chunk$set(echo = F)

0.1 Load data:

The permutation takes to long to allow for re-computing the code each time, so load it:

1. Objective measures

1.1. Consistency:

The mean area is calculated by adding together the area for each stimulus and dividing by 29. This unit is transformed into a percentage area taking into account the different pixel resolution of each participant.
Mean area = $(Summed area / 29) * 100 / Screen area $ Where: \(Screen area = Xpixels * Ypixels\)

Note: 29 since 29 stimuli

1.2 Permuted consistency

Replicate Rothen methods. Might take some time to compute.

Calculating chance levels of consistency To create permuted datasets for each participant: the 87 xy coordinates are randomly shuffled so they are no longer linked to the original data labels (“Monday”, “5”, “April”, etc.). The mean area of the triangles based on the shuffled coordinates is computed (as described above), and the whole process is repeated 1000 times to obtain a subject-specific distribution of chance levels of consistency. A z-score is calculated comparing the observed consistency against the mean and SD of the permuted data: \(Z = [(observed consistency) – (mean consistency of permuted data)] / (SD of permuted data)\)

Code retrieved from OSF (adapted here):

1.3. SD as in Ward

As in Ward:

“Specifically, the standard deviation of the x-coordinates and/or the standard deviation of the y-coordinates (measured across all trials) should exceed a proposed value of 0.075 for a normalized screen with width and height of 1 unit.”

“A participant who produced a horizontal straight-line form would have a very low standard deviation in the y-coordinates but a high standard deviation in x-coordinates, and a participant with a vertical line would have the reverse profile. A participant with a circular spatial form would be high on both. A participant who clicks randomly around the screen would also be high on both x and y standard deviation, but would fail the consistency tests (the triangles would be large).”

2. Line intersection

An idea I have is to look into the lines and order of the forms. I would exclude when lines crosses. (since we expect forms the lines crossing means no form is formed). Needs refinement.

I think that the number of stimuli per condition should be taken into account (i.e. 9 numbers, 7 days, 12 months). Hence would need to be divided by this number of stimulus.

In each condition the connected x and y generates a segment, hence the number of segment is length(stimuli)-1. Moreover, currently, each stimuli is connected by 3 segment, one for each (of the 3) repetition. So dividing by 3, we have the average number of segment corssings per condition. Next we sum these for each ID Ideally we should compute the number of crossings across the repetitions, in addition to make it more complex it would also be computationally more demanding, and I don’t beleive it would lead to a significant difference.

To do: - Maybe the easier would be to have the average number of crossing per segment.

IMPORTANT: data frame needs to be informed of stimulus order to make sense!

´r apa_print(t.test(GA_lineInter ~ SynQuest_NR, ds))$full_result`

´r apa_print(t.test(Consistency ~ SynQuest_NR, ds))$full_result`

3. ROC to set thresholds

We consider the questionnaires as ground truth. Since the questionnaires are scalable we still need to determine a different thresholds for the questionnaire. Also some questions might be more relevant than others to “diagnose” synesthesia (see Syn_NR).

3.1. ROC Segment intersections

3.1.1. AUC

We computed the ROC between the questionnaire with NR selected questions and the per participants averaged segment interceptions.

4.1.2. ROC and threshold

Form the ROC we used Youden’s J statistic o calculate the best threshold. This statistic maximizes the difference between true positive rate (i.e. sensitivity) and false positive rate (i.e. specificity - 1). This led to a threshold of 0.09 with a sensitivity of 84.15 and a specificity of 56.88. See Figure X:

3.1.3. Cumulative density plot

Visualize it with cumulative empirical density plots. In this plots the x xis indicates the average number of intersections per segments. The y axis indicates the percentage of participant. Hence while 93 % of the self identified synesthetes have less than 0.0927958` intersection per segment, 56 % of not self-identified synesthetes do.

3.1.2 For months

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_NR ~ nLineCross, data = ., percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: nLineCross in 13860 controls (SynQuest_NR FALSE) > 2952 cases (SynQuest_NR TRUE).
## Area under the curve: 70.91%
## 95% CI: 70.19%-71.63% (DeLong)

3.1.2 For weekdays

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_NR ~ nLineCross, data = ., percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: nLineCross in 8085 controls (SynQuest_NR FALSE) > 1722 cases (SynQuest_NR TRUE).
## Area under the curve: 64.38%
## 95% CI: 63.32%-65.45% (DeLong)

3.1.2 For nubmers

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_NR ~ nLineCross, data = ., percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: nLineCross in 11550 controls (SynQuest_NR FALSE) > 2460 cases (SynQuest_NR TRUE).
## Area under the curve: 69.02%
## 95% CI: 68.18%-69.85% (DeLong)

3.2. ROC Consistency

3.2.1. AUC

3.2.2. ROC and threshold

Form the ROC we used Youden’s J statistic o calculate the best threshold. This statistic maximizes the difference between true positive rate (i.e. sensitivity) and false positive rate (i.e. specificity - 1). This led to a threshold of 0.16 with a sensitivity of 80.49 and a specificity of 40.26. See Figure X:

3.2.3. Cumulative density plot

Visualize it with cumulative empirical density plots. In this plots the x xis indicates the average number of intersections per segments. The y axis indicates the percentage of participant. Hence while 93 % of the self identified synesthetes have less than 1 intersection per segment, 52 % of not self-identified synesthetes do.

### 3.1.2 For months

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls < cases

## 
## Call:
## roc.formula(formula = SynQuest_NR ~ triangle_area, data = .,     percent = TRUE, ci = TRUE, boot.n = 100, ci.alpha = 0.9,     stratified = FALSE, plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: triangle_area in 13860 controls (SynQuest_NR FALSE) < 2952 cases (SynQuest_NR TRUE).
## Area under the curve: 52.65%
## 95% CI: 51.68%-53.62% (DeLong)

3.1.2 For weekdays

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls < cases

## 
## Call:
## roc.formula(formula = SynQuest_NR ~ triangle_area, data = .,     percent = TRUE, ci = TRUE, boot.n = 100, ci.alpha = 0.9,     stratified = FALSE, plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: triangle_area in 8085 controls (SynQuest_NR FALSE) < 1722 cases (SynQuest_NR TRUE).
## Area under the curve: 53.12%
## 95% CI: 51.85%-54.4% (DeLong)

3.1.2 For nubmers

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls < cases

## 
## Call:
## roc.formula(formula = SynQuest_NR ~ triangle_area, data = .,     percent = TRUE, ci = TRUE, boot.n = 100, ci.alpha = 0.9,     stratified = FALSE, plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: triangle_area in 11550 controls (SynQuest_NR FALSE) < 2460 cases (SynQuest_NR TRUE).
## Area under the curve: 51.3%
## 95% CI: 50.26%-52.35% (DeLong)

3.3. Power

## 
##      Two ROC curves power calculation 
## 
##          ncases = 7134
##       ncontrols = 33495
##            auc1 = 0.7179126
##            auc2 = 0.5265125
##       sig.level = 0.05
##           power = 1
##     alternative = two.sided

4. Add oder Criteris

4.1. Questinnnaire Ward

The same but with the Questionnaire threshold as defined in Ward. This led to a threshold of 0.25 with a sensitivity of 0.85 and a specificity of 0.62.

4.2. Permuted z-score consistency

Replicate Rothen methods. Might take some time to compute.

Calculating chance levels of consistency To create permuted datasets for each participant: the 87 xy coordinates are randomly shuffled so they are no longer linked to the original data labels (“Monday”, “5”, “April”, etc.). The mean area of the triangles based on the shuffled coordinates is computed (as described above), and the whole process is repeated 1000 times to obtain a subject-specific distribution of chance levels of consistency. A z-score is calculated comparing the observed consistency against the mean and SD of the permuted data: \(Z = [(observed consistency) – (mean consistency of permuted data)] / (SD of permuted data)\)

Code retrieved from OSF (adapted here):

see ´SynPermzs´

Note this might vary due to the randomisation process involved in the permutation. Some sd are 0, leadiong to NaN in the z score. The two additional lines are to take this into account.

4.3. SD threshold

With this criteria alone 93.3618844 % would qualify as 3S

5. Sankey plot for each criteria:

6. NR Data

6.1. Consistency

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

##   threshold sensitivity specificity
## 1 0.2002146    84.84848    70.27027

6.2. Line Intersections

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

##    threshold sensitivity specificity
## 1 0.02574597    45.45455    70.27027

Weeks

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = Syn ~ GA_lineInter, data = ., percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 777 controls (Syn FALSE) > 693 cases (Syn TRUE).
## Area under the curve: 46.81%
## 95% CI: 43.74%-49.87% (DeLong)

Months

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = Syn ~ GA_lineInter, data = ., percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 1332 controls (Syn FALSE) > 1188 cases (Syn TRUE).
## Area under the curve: 46.81%
## 95% CI: 44.47%-49.15% (DeLong)

Numbers

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = Syn ~ GA_lineInter, data = ., percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 1110 controls (Syn FALSE) > 990 cases (Syn TRUE).
## Area under the curve: 46.81%
## 95% CI: 44.24%-49.37% (DeLong)

6.2.1.

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

##      threshold sensitivity specificity
## 1 1.312927e-05    70.83333    81.08108

7. Bonus ROC for each questionnaire Criteria

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 38454 controls (SynQuest_2 FALSE) > 8265 cases (SynQuest_2 TRUE).
## Area under the curve: 76.53%
## 95% CI: 76.07%-76.99% (DeLong)
##    threshold sensitivity specificity
## 1 0.09279577    89.47368    57.01357

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 36801 controls (SynQuest_2 FALSE) > 9918 cases (SynQuest_2 TRUE).
## Area under the curve: 76.52%
## 95% CI: 76.08%-76.97% (DeLong)
##    threshold sensitivity specificity
## 1 0.09311506    88.59649    58.62884

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 34887 controls (SynQuest_2 FALSE) > 11832 cases (SynQuest_2 TRUE).
## Area under the curve: 73.74%
## 95% CI: 73.28%-74.21% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    88.97059    53.86534

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 33321 controls (SynQuest_2 FALSE) > 13398 cases (SynQuest_2 TRUE).
## Area under the curve: 74.34%
## 95% CI: 73.89%-74.8% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563    88.31169    55.61358

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 31842 controls (SynQuest_2 FALSE) > 14877 cases (SynQuest_2 TRUE).
## Area under the curve: 75.25%
## 95% CI: 74.81%-75.69% (DeLong)
##   threshold sensitivity specificity
## 1 0.1404563     87.7193    57.37705

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 30276 controls (SynQuest_2 FALSE) > 16443 cases (SynQuest_2 TRUE).
## Area under the curve: 74.32%
## 95% CI: 73.88%-74.76% (DeLong)
##   threshold sensitivity specificity
## 1  0.135609    84.65608    58.90805

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 29493 controls (SynQuest_2 FALSE) > 17226 cases (SynQuest_2 TRUE).
## Area under the curve: 73.94%
## 95% CI: 73.49%-74.38% (DeLong)
##   threshold sensitivity specificity
## 1 0.1494543    85.35354    57.52212

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 28275 controls (SynQuest_2 FALSE) > 18444 cases (SynQuest_2 TRUE).
## Area under the curve: 73.11%
## 95% CI: 72.66%-73.56% (DeLong)
##   threshold sensitivity specificity
## 1 0.1494543    83.49057    58.15385

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 26796 controls (SynQuest_2 FALSE) > 19923 cases (SynQuest_2 TRUE).
## Area under the curve: 72.56%
## 95% CI: 72.11%-73.02% (DeLong)
##   threshold sensitivity specificity
## 1  0.135609    79.91266    61.03896

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 25404 controls (SynQuest_2 FALSE) > 21315 cases (SynQuest_2 TRUE).
## Area under the curve: 72.19%
## 95% CI: 71.73%-72.65% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    84.89796    54.79452

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 24273 controls (SynQuest_2 FALSE) > 22446 cases (SynQuest_2 TRUE).
## Area under the curve: 71.94%
## 95% CI: 71.48%-72.41% (DeLong)
##   threshold sensitivity specificity
## 1 0.2460815    84.10853    55.91398

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 23229 controls (SynQuest_2 FALSE) > 23490 cases (SynQuest_2 TRUE).
## Area under the curve: 71.8%
## 95% CI: 71.33%-72.26% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    82.96296    57.30337

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 22011 controls (SynQuest_2 FALSE) > 24708 cases (SynQuest_2 TRUE).
## Area under the curve: 71%
## 95% CI: 70.52%-71.47% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    81.33803    57.70751

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 20880 controls (SynQuest_2 FALSE) > 25839 cases (SynQuest_2 TRUE).
## Area under the curve: 70.9%
## 95% CI: 70.42%-71.38% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    83.16498    55.41667

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 20358 controls (SynQuest_2 FALSE) > 26361 cases (SynQuest_2 TRUE).
## Area under the curve: 70.15%
## 95% CI: 69.67%-70.64% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    82.17822    55.12821

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 19314 controls (SynQuest_2 FALSE) > 27405 cases (SynQuest_2 TRUE).
## Area under the curve: 68.45%
## 95% CI: 67.95%-68.95% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956          80    54.05405

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 18096 controls (SynQuest_2 FALSE) > 28623 cases (SynQuest_2 TRUE).
## Area under the curve: 67.7%
## 95% CI: 67.19%-68.21% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    78.41945    53.84615

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 17139 controls (SynQuest_2 FALSE) > 29580 cases (SynQuest_2 TRUE).
## Area under the curve: 67.23%
## 95% CI: 66.71%-67.74% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    77.35294    53.80711

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 15747 controls (SynQuest_2 FALSE) > 30972 cases (SynQuest_2 TRUE).
## Area under the curve: 67.94%
## 95% CI: 67.42%-68.45% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    76.40449    54.69613

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 15138 controls (SynQuest_2 FALSE) > 31581 cases (SynQuest_2 TRUE).
## Area under the curve: 67.86%
## 95% CI: 67.35%-68.38% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    76.03306    55.17241

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 14529 controls (SynQuest_2 FALSE) > 32190 cases (SynQuest_2 TRUE).
## Area under the curve: 66.82%
## 95% CI: 66.3%-67.35% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    72.16216    57.48503

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 14007 controls (SynQuest_2 FALSE) > 32712 cases (SynQuest_2 TRUE).
## Area under the curve: 66.12%
## 95% CI: 65.58%-66.65% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    71.54255    57.14286

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 13398 controls (SynQuest_2 FALSE) > 33321 cases (SynQuest_2 TRUE).
## Area under the curve: 65.41%
## 95% CI: 64.87%-65.95% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    70.75718    56.49351

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 13050 controls (SynQuest_2 FALSE) > 33669 cases (SynQuest_2 TRUE).
## Area under the curve: 64.67%
## 95% CI: 64.12%-65.22% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    70.28424          56

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 12267 controls (SynQuest_2 FALSE) > 34452 cases (SynQuest_2 TRUE).
## Area under the curve: 63.59%
## 95% CI: 63.02%-64.16% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    69.69697    56.02837

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 11223 controls (SynQuest_2 FALSE) > 35496 cases (SynQuest_2 TRUE).
## Area under the curve: 61.49%
## 95% CI: 60.91%-62.08% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    68.62745    55.03876

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 10353 controls (SynQuest_2 FALSE) > 36366 cases (SynQuest_2 TRUE).
## Area under the curve: 59.65%
## 95% CI: 59.05%-60.26% (DeLong)
##   threshold sensitivity specificity
## 1 0.2277952    67.70335    53.78151

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 10092 controls (SynQuest_2 FALSE) > 36627 cases (SynQuest_2 TRUE).
## Area under the curve: 58.83%
## 95% CI: 58.21%-59.44% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    70.30879          50

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 9744 controls (SynQuest_2 FALSE) > 36975 cases (SynQuest_2 TRUE).
## Area under the curve: 57.58%
## 95% CI: 56.95%-58.2% (DeLong)
##   threshold sensitivity specificity
## 1  0.332956    69.64706    48.21429

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 9222 controls (SynQuest_2 FALSE) > 37497 cases (SynQuest_2 TRUE).
## Area under the curve: 55.65%
## 95% CI: 55.01%-56.28% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    67.05336    48.11321

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 8787 controls (SynQuest_2 FALSE) > 37932 cases (SynQuest_2 TRUE).
## Area under the curve: 54.59%
## 95% CI: 53.95%-55.24% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    66.74312    47.52475

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 8352 controls (SynQuest_2 FALSE) > 38367 cases (SynQuest_2 TRUE).
## Area under the curve: 53.66%
## 95% CI: 53.01%-54.31% (DeLong)
##   threshold sensitivity specificity
## 1 0.2677348    66.21315    45.83333

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 7569 controls (SynQuest_2 FALSE) > 39150 cases (SynQuest_2 TRUE).
## Area under the curve: 50.58%
## 95% CI: 49.92%-51.25% (DeLong)
##     threshold sensitivity specificity
## 1 0.006269592    18.66667    89.65517

## Setting levels: control = FALSE, case = TRUE
## Setting direction: controls > cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 7308 controls (SynQuest_2 FALSE) > 39411 cases (SynQuest_2 TRUE).
## Area under the curve: 50.5%
## 95% CI: 49.82%-51.18% (DeLong)
##   threshold sensitivity specificity
## 1 0.3077325    66.22517    41.66667

## Setting levels: control = FALSE, case = TRUE

## Setting direction: controls < cases

## 
## Call:
## roc.formula(formula = SynQuest_2 ~ GA_lineInter, data = ds, percent = TRUE,     ci = TRUE, boot.n = 100, ci.alpha = 0.9, stratified = FALSE,     plot = TRUE, auc.polygon = TRUE, max.auc.polygon = TRUE,     grid = TRUE, print.auc = TRUE, show.thres = TRUE)
## 
## Data: GA_lineInter in 6699 controls (SynQuest_2 FALSE) < 40020 cases (SynQuest_2 TRUE).
## Area under the curve: 51.7%
## 95% CI: 51%-52.39% (DeLong)
##   threshold sensitivity specificity
## 1  1.523018    23.91304    90.90909

## R version 4.3.3 (2024-02-29)
## Platform: aarch64-apple-darwin20 (64-bit)
## Running under: macOS 15.5
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## time zone: Europe/Zurich
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] pROC_1.18.5       ggalluvial_0.12.5 ggridges_0.5.6    ggplot2_3.5.2    
##  [5] papaja_0.1.3      tinylabels_0.2.5  dplyr_1.1.4       tidyr_1.3.1      
##  [9] readxl_1.4.5      readr_2.1.5      
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.6       jsonlite_2.0.0     compiler_4.3.3     Rcpp_1.0.14       
##  [5] tidyselect_1.2.1   jquerylib_0.1.4    scales_1.4.0       yaml_2.3.10       
##  [9] fastmap_1.2.0      plyr_1.8.9         R6_2.6.1           labeling_0.4.3    
## [13] generics_0.1.4     knitr_1.50         tibble_3.3.0       RColorBrewer_1.1-3
## [17] bslib_0.9.0        pillar_1.10.2      tzdb_0.5.0         rlang_1.1.6       
## [21] cachem_1.1.0       xfun_0.52          sass_0.4.10        cli_3.6.5         
## [25] withr_3.0.2        magrittr_2.0.3     digest_0.6.37      grid_4.3.3        
## [29] rstudioapi_0.17.1  hms_1.1.3          lifecycle_1.0.4    vctrs_0.6.5       
## [33] evaluate_1.0.3     glue_1.8.0         farver_2.1.2       cellranger_1.1.0  
## [37] rmarkdown_2.29     purrr_1.0.4        tools_4.3.3        pkgconfig_2.0.3   
## [41] htmltools_0.5.8.1